Open Access 



Research 



BMJ Open Development of evidence-based 
Australian medication-related 
indicators of potentially preventable 
hospitalisations: a modified RAND 
appropriateness method 

Gillian E Caughey,'' Lisa M Kalisch Ellett,'' Te Ying Wong^ 



To cite: Caughey GE, Kalisch 
Ellett LM, Wong TY. 
Development of evidence- 
based Australian medication- 
related indicators of 
potentially preventable 
hospitalisations: a modified 
RAND appropriateness 
method. BMJ Open 201 4;4: 
e004625. doi:10.1136/ 
bmjopen-201 3-004625 

► Prepublication history for 
this paper is available online. 
To view these files please 
visit the journal online 
(http://dx.doi.org/10.1136/ 
bmjopen-201 3-004625). 



Received 4 December 2013 
Revised 17 March 2014 
Accepted 1 April 2014 



CrossMark 



For numbered affiliations see 
end of article. 



Correspondence to 

Dr Lisa Kalisch Ellett; 
lisa.kalisch@unisa.edu.au 



ABSTRACT 

Objective: Indicators of potentially preventable 
hospitalisations have been adopted internationally as a 
measure of health system performance; however, few 
assess appropriate processes of care around 
medication use, that if followed may prevent 
hospitalisation. The aim of this study was to develop 
and validate evidence-based medication-related 
indicators of potentially preventable hospitalisations. 
Setting: Australian primary healthcare. 
Participants: Medical specialists, general practitioners 
and pharmacists. A modified RAND appropriateness 
method was used for the development of medication- 
related indicators of potentially preventable 
hospitalisations, which included a literature review, 
assessment of the strength of the supporting evidence 
base, an initial face and content validity by an expert 
panel, followed by an independent assessment of 
indicators by an expert clinical panel across various 
disciplines, using an online survey. 
Primary outcome measure: Analysis of ratings was 
performed on the four key elements of preventability; 
the medication-related problem must be recognisable, 
the adverse outcomes foreseeable and the causes and 
outcomes identifiable and controllable. 
Results: A total of 48 potential indicators across all 
major disease groupings were developed based on 
level III evidence or greater, that were independently 
assessed by 78 expert clinicians (22.1% response 
rate). The expert panel considered 29 of these (60.4%) 
sufficiently valid. Of these, 21 (72.4%) were based on 
level I evidence. 

Conclusions: This study provides a set of face and 
content validated indicators of medication-related 
potentially preventable hospitalisations, linking 
suboptimal processes of care and medication use with 
subsequent hospitalisation. Further analysis is required 
to establish operational validity in a population-based 
sample, using an administrative health database. 
Implementation of these indicators within routine 
monitoring of healthcare systems will highlight those 
conditions where hospitalisations could potentially be 
avoided through improved medication management. 



Strengths and limitations of this study 



■ The clinical indicators developed were based on 
high-level evidence together with expert clinical 
panel assessment. 

■ Since the clinical indicators were developed 
using Australian-specific resources they may 
need to be adapted for use in other settings. 

■ This study provides a set of face and content 
validated indicators of potentially preventable 

I hospitalisations, linking to suboptimal processes 
" of care and medication use with subsequent 
hospitalisation. 



INTRODUCTION 

Clinical indicators of potentially preventable 
hospitalisations are used as a measure of 
health system performance and quality of 
healthcare provided to patients/ ^ Potentially 
preventable hospitalisations are defined as 
those hospitalisations that could be prevented 
with the provision of timely and effective 
primary care.^ Medication-related hospitalisa- 
tions are relatively common. A literature 
review found that 2-3% of all hospital admis- 
sions in the Australian healthcare setting were 
medication related, with half considered to be 
potentially preventable.^ A systematic review of 
studies from around the world found that a 
median of 3.7% of all hospital admissions were 
preventable medication-related admissions.^ 
The identification and subsequent reduction 
of the most common medication-related 
potentially preventable hospitalisations will 
improve morbidity and quality of life for 
patients, safety of the healthcare system and 
reduce healthcare expenditure. 

Clinical indicators of medication-related 
potentially preventable hospitalisations have 
been developed which link suboptimal 
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processes of care with medicine use to assess adverse 
outcomes including hospitalisation.^ The overall inci- 
dence of preventable medication-related hospitalisations 
when measured using these clinical indicator sets has 
been reported to range between 3% and 20%, depend- 
ing on the country of the study population and the clin- 
ical indicator set used.^~^^ Using the previously 
developed clinical indicators, the prevalence of poten- 
tially preventable medication-related hospitalisations in 
the Australian healthcare setting between 1 January 
2004 and 31 December 2008 was examined. During the 
5-year study period there were 44 416 (20.5%) poten- 
tially preventable medication-related hospitalisations, 
equating to 9000 preventable admissions each year.^^ 
However, in undertaking the study, it became apparent 
that many of the internationally developed indicators 
were not relevant or applicable to the Australian health- 
care setting. A cross-country comparison between the 
USA and the UK indicators found that of the 46 indica- 
tors assessed, 58% were relevant to the USA but not the 
UK, and only 41% were deemed to be relevant in the 
healthcare setting of both countries.^ Given the signifi- 
cant differences between the USA and the UK health- 
care systems to that of Australia, there may be additional 
indicators, specifically relevant to the Australian health- 
care system, that should be examined. Further, the inter- 
national indicators were developed over 10 years ago 
and there are likely to be a number of indicators based 
on new medicines introduced since then. 

Prior studies which developed clinical indicators for 
potentially preventable medication-related hospitalisations 
used the Delphi technique, which measures con- 

sensus among experts using a series of structured 
surveys. Recent studies have highlighted the need for 
clinical indicators to be evidence based, rather than based 
on expert consensus only.^^ Increasingly, the RAND 
appropriateness method is used in indicator develop- 
ment,^^ which develops indicators by combining 
evidence-based recommendations from clinical guidelines 
with expert clinical opinion. In addition, recent studies 
have highlighted the need for clinical indicators to be 
country specific to reflect current practice within individ- 
ual healthcare systems. To date, no evidence-based 
indicators of medication-related potentially preventable 
hospitalisations have been developed specific for the 
Australian setting. The aim of this study was to develop 
and validate Australian evidence-based medication-related 
indicators of potentially preventable hospitalisations. 

METHODS 

A modified RAND appropriateness method was used for 
the development of medication-related indicators of 
potentially preventable hospitalisations, which has 
characteristics of both the Delphi and Nominal Group 
Techniques, providing a systematic method to combine 
evidence with expert opinion. It consists of a literature 
review, assessment of the strength of the supporting 



evidence base, an initial face and content validity assess- 
ment by an expert panel, followed by an independent 
assessment of indicators by an expert clinical panel 
across various disciplines, using an online survey. 

Identification of existing indicators and development 
of new indicators 

A number of methods were used to systematically iden- 
tify and develop clinical indicators for medication- 
related potentially preventable hospitalisations, specific 
for the Australian healthcare setting. A literature review 
was conducted to identify all published studies of indica- 
tors for preventable medication-related hospitalisations 
that could be adapted using specific inclusion criteria. 
Identification and development of additional clinical 
indicators was based on chronic diseases included in 
Australia's National Health Priority Areas. In addition, 
indicators were developed for gastrointestinal disorders, 
which are associated with high prevalence and morbidity 
in Australia.^^ Australian treatment and clinical guide- 
lines for these chronic conditions were then examined 
to identify potential mediation-related issues relevant for 
the development of clinical indicators for preventable 
medication-related hospitalisations. 

Literature review to identify existing clinical indicators of 
potentially preventable medication-related hospitalisations 

A literature review of all published studies on clinical 
indicators for preventable medication-related hospitalisa- 
tions was conducted from January 2001 to December 
2012, inclusive. The primary search terms used were 
'indicators', 'prevent$ OR avoid$', 'medication OR 
drug-related', 'hospitalisation OR morbidity' and 
'adverse drug event'. MEDLINE (via Ovid) and 
EMBASE were searched, with results limited to articles 
published in English and conducted in adults. Studies 
which developed indicators not associated with the 
outcome of hospitalisation were excluded. Reference 
lists of relevant identified studies were further searched 
to identify additional papers. The following information 
was extracted from each suitable study: the hospitalisa- 
tion outcome, the process of care leading to the 
outcome and references (ie, studies which developed 
the indicator). Clinical indicators were grouped accord- 
ing to broader chronic disease groupings and similar 
clinical indicators obtained from different studies were 
recorded as one clinical indicator. 

Predefined inclusion criteria were used to determine 
the applicability and relevance of previously published 
clinical indicators to the Australian healthcare setting. 
Indicators that did not meet one or more of these cri- 
teria were excluded from the study. 

The inclusion criteria were the following: 
1. Strength of supporting evidence must be Grade B or 
level III or higher, based on the National Health and 
Medical Research Council (NHMRC) evidence matrix.^^ 
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2. The medicine must be available in Australia and sub- 
sidised under the Schedule of Pharmaceutical 
Benefits (PBS or RPBS).^^ 

3. The process of care must concur with Australian 
treatment guidelines. 

4. The process of care can be identified in Australian 
electronic health records. 

Concordance of the indictors with current Australian 
treatment and clinical guidelines were identified from 
the Australian Therapeutic Guidelines,^"^ Australian 
Medicines Handbook^^ and clinical guidelines including 
cardiovascular disease,^^"^^ respiratory conditions,^^ 
diabetes,^^ musculoskeletal conditions'^ and 
mental health.'^ Where the international clinical indi- 
cators differed slightly from Australian guidelines, modi- 
fications were developed if appropriate. The guidelines 
were also searched to determine the level of supporting 
evidence. The strength of the supporting evidence for 
each indicator was assessed and categorised into five 
levels based on current Australian standards, used for 
guideline development.'^ Only those indicators with 
level III or greater evidence were included. The WHO 
International Classification of Diseases (ICD) 10-AM 
classification was used to identify codes for hospitalisa- 
tion outcomes. 

Development of new clinical indicators of potentially 
preventable medication-related hospitalisations 

Development of new clinical indicators was largely based 
on those chronic diseases included in Australia's 
National Health Priority Areas. Chronic obstructive 
pulmonary disease was included with asthma under the 
broad disease category of respiratory conditions due to 
its large disease burden and mortality."^^ Gastrointestinal 
disorders which are associated with high prevalence and 
morbidity in Australia, were also included.^^ Clinical 
indicators for cancer were not developed in this study. 
Medicine use for cancer is highly specialised and varied 
depending on the type of cancer, and the development 
of new medicines for these conditions is a fast evolving 
area. To develop new indicators treatment and clinical 
guidelines for each of the conditions were reviewed, 
with a focus on treatment considerations, medicine class 
statements and monographs, contraindications, precau- 
tions, recommended testing and follow-up. All newly 
developed indicators were required to meet the inclu- 
sion criteria used for previously published clinical indi- 
cators, as described above. 

Initial face and content validity by a convenience sample 
of pharmacists 

An initial face and content validity of the compiled list 
of indicators was undertaken with a convenience sample 
of eight clinical pharmacists. Based on the four elements 
of preventability developed by Hepler and Strand,"^^ they 
were asked the following questions: would you expect 
most health professionals to 

1. Recognise the problem in the process of care? 



2. Foresee the potential for hospitalisation associated 
with the process of care? 

3. Know how to change the process of care to reduce 
the likelihood of hospitalisation? 

4. Be able to change the process of care to reduce the 
likelihood of hospitalisation? 

Responses to each of the four elements of preventabil- 
ity were rated on a three-point Likert scale, where T 
indicates disagreement, '2' uncertain or equivocal and 
'3' agreement, together with comments to allow for 
feedback or suggestions regarding specific elements or 
readability. For each indicator, a majority agreement 
(5/8 or 62.5%) by the convenience sample across all 
four elements of preventability was required for inclu- 
sion in our final list for validation by an expert panel. 

Expert panel assembly, survey and analysis 

The final list of indicators for validation were grouped 
into subject categories (cardiovascular disease, diabetes, 
renal, mental health, respiratory, gastrointestinal and 
osteoporosis/fracture indicators) and sent to clinical 
experts for review. Experts were identified as clinical 
leaders in their field, that included both medical physi- 
cians (general practitioners and specialists) and pharma- 
cists (including certified geriatric pharmacists and 
clinical pharmacists), across Australia from a range of 
healthcare settings. A total of 352 clinical experts were 
identified and contacted to be part of the expert clinical 
panel for validation of the indicators, between December 
2012 and March 2013. They were invited to score the 
indicators using an online survey (SurveyMonkey http:// 
www.surveymonkey.com) on the four elements of prevent- 
ability, as described above. Participants were not able to 
respond to the survey more than once. A brief summary 
of each indicator was provided; the level of evidence for 
each indicator together with the reference (s) supporting 
the level of evidence. A priori criteria of consensus for 
validation for each of the indicators were defined; an 
average score of 70% or greater agreement by the expert 
panellists, across all four elements of preventability for 
each indicator, were deemed to meet requirements for 
validation of an indicator. As described above, responses 
to each of the four elements of preventability were rated 
on a three-point Likert scale, where T indicates disagree- 
ment, '2' uncertain or equivocal and '3' agreement, 
together with comments to allow for feedback on each of 
the individual indicators. 

RESULTS 

A total of 48 potential indicators across major disease 
groupings based on level III evidence or greater were 
developed (table 1), all of which had majority agree- 
ment in the initial face and content validity by a conveni- 
ence sample of eight clinical pharmacists. Of these, 13 
were from previously developed medication-related indi- 
cators of potentially preventable hospitalisations, 15 were 
modified to be applicable to the Australian healthcare 
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Table 1 Australian medication-related potentially preventable hospitalisation clinical indicator set 



I 

I 

I 

2 

I 

3 

I 

5 
6 

i 

9 

I 



Source of indicator 



Hospitalisation 
outcome 



Process of care (preceding hospitalisation) 



Level of evidence 



Previously 
published; not 
modified 



Previously published; 
modified for this study 



Newly 
developed 



Cardiovascular indicators 
Acute coronary 
syndrome 

Acute coronary 
syndrome 



CHF 
CHF 

CHF 

CHF 

CHF or cardiac 

ischaemic event 

CHF and / or heart block 



CHF or Ml 
Ischaemic stroke 



VTE or stroke 



Mental health indicators 
12 Bipolar disorder 



Acute confusion 



Acute confusion 



History of Ml (in 2 years prior to admission) 

Not on aspirin, p-blocker, ACEI or ARB and statin (in 

3 months prior to admission) 

Patient has coronary artery stent (in 1 year prior to 

admission) 

No use of aspirin or clopidogrel (in 12 months prior to 
admission) 

History of CHF (in 2 years prior to admission) 

Not on an ACEI or ARB (in 3 months prior to admission) 

History of CHF (in 2 years prior to admission) 

Not on a heart failure indicated p-blocker (in 3 months 

prior to admission) 

History of CHF 

Use of rosiglitazone or pioglitazone (in 6 months prior to 

admission) 

History of CHF 

Use of NSAID (in 3 months prior to admission) 

History of IHD (in 2 years prior to admission) 

Use of rosiglitazone (in 6 months prior to admission) 

History of CHF with heart block or advanced bradycardia 

(in 2 years prior to admission) 

Use of digoxin (in 6 months prior to admission) 

Concurrent use of insulin and rosiglitazone 

History of chronic AF or ischaemic stroke in 2 years prior 

to admission) 

No use of warfarin or aspirin (in 3 months prior to 
admission) 

History of coronary artery disease or VTE 
Use of raloxifene 

History of bipolar disorder 
Use of lithium 

Drug level not monitored in the previous 3 months 
Patient aged >65 years 

Use of two or more agents with anticholinergic activity 
OR use of an agent with high anticholinergic activity 
Patient aged >65 years 
Use of multiple psychotropic medications (eg, 
benzodiazepines, tricyclic antidepressants) 



Aspirin, p-blocker — 
level I, ACEI/ARB, 
statin— level 11^^ 
Level 



Level 1^^ 
Level 1^^ 



Level 

Level 1^^ 
Level l^° 
Level 111^^ 

Level 111^° 
Level l^° 

Level 11^^ 
Level 1^^ 
Level 111^^ 
Level 111^^ 



Changed outcome from just 
Ml; added ACEI/ARB and 
statin^ ^ 



Added ARB for those 
intolerant to ACEI^"^ 



Removed fluid overload 



from outcome' 



2 6-8 



Continued 



"able 1 Continued 



Source of indicator 



Hospitalisation 
Number outcome 



Process of care (preceding hospitalisation) 



Level of evidence 



Previously 
published; not 
modified 



Previously published; 
modified for this study 



Newly 
developed 



Serotonin toxicity 



Serotonin toxicity 



Use of duloxetine, fentanyl, tramadol, SSRIs, TCAs or 
venlafaxine concurrently with MAO! or moclobemide, or 
within 14 days of stopping MAO! 
Concurrent treatment with strong CYP1A2 inhibitors (eg, 
duloxetine) with fluvoxamine 



Respiratory indicators 
|17 Asthma or COPD 

Asthma 



COPD 

Asthma or COPD 

Influenza-related 
pneumonia 

Pneumococcal 
pneumonia or sepsis 



20 



21 
22 



Gl indicators 



History of asthma or COPD 

Use of a p-blocker eye drops for glaucoma 

History of asthma 

Use of SABA more than 3 times/week or use of LABA 

No use of inhaled corticosteroids 

Moderate to severe COPD with frequent exacerbation 

Use of long-acting p-agonist or anticholinergic 

No use of inhaled corticosteroids 

History of asthma or COPD 

No contraindication to influenza vaccine 

No influenza vaccination in the previous year 

Patient aged >65 

No contraindication to influenza vaccine 
No influenza vaccine in the previous year 
Patient aged >65 

No contraindication to pneumococcal vaccine 
No pneumococcal vaccine in the previous 6 years 



Level \\r 

Level llP^ 

Level 1^^ 
Level 1^^ 

Level 1^^ 

Level 1^^ ^ 

Level 1^^ ' 

Level 111^^ 



Asthma only, Australian 
guideline specific''"^ 

COPD only, Australian 
guideline specific^"^ 



23 


Gl bleed, perforation or 


1. 


History of Gl ulcer or bleeding 


Level 11^^ ^9 


Added gastroprotective 






ulcer or gastritis 


2. 


NSAID use for at least 1 month 




agent^^ ""^ 




^24 




3. 


No use of gastroprotective agent (eg, PPI) 


Level 1^° 








Chronic constipation or 
impaction 


1. 


Use of two or more agents with low-to-moderate 
anticholinergic activity; OR use of a highly anticholinergic 


2 










agent 


Level 1^^ 


8 




25 


Chronic constipation or 


1. 


Regular use of a strong opioid analgesic (fentanyl. 






impaction 




oxycodone, morphine) 










2. 


No concurrent use of a laxative 








26 


Gl ulcer 


1. 


Patient with dyspepsia 


Level 1^^ 




25 






2. 


PPI not prescribed 






25 


27 


Gl ulcer 


1. 
2. 


Patient with a positive test for Helicobacter pylori 
Not prescribed H pylori eradication therapy 


Level 1^^ 




28 


Gl ulcer or bleed 


1. 


Patient with osteoarthritis 


Level 1^^ 




36 






2. 


Dispensed long-term NSAIDs (including COX-2) therapy 


Level 1^^ 




25 


29 


Oesophagitis, 
oesophageal ulceration 
or stricture 


1. 
2. 


History of oesophageal disorders (active oesophagitis, 
oesophageal ulceration, stricture or achalasia) 
Use of alendronate 





Continued 



"able 1 Continued 



Source of indicator 



Hospitalisation 
Number outcome 



Process of care (preceding hospitalisation) 



Level of evidence 



Previously 

published; not Previously published; 
modified modified for this study 



Newly 
developed 



Osteoporosis/fracture indicators 
30a Osteoporosis or fracture 




30b 
31 



32 



Osteoporosis or fracture 
Fracture 



Fracture 



Fracture 



Fracture 



Fracture 



37 



Arrhythmia 
Hypercalcaennia 



Renai indicators 

38 Renal failure or 

II nephropathy 
5 



r59 
40 
41 



Renal failure 
Renal failure 
Urinary retention 



Use of systemic corticosteroids for at least 3 months Level 1^^ 

No osteoporosis prophylaxis (women: no use of HRT, 
bisphosphonate, teriparatide, selective oestrogen receptor 
modulators or strontium; men: no use of bisphosphonate 
or teriparatide) 

This indicator is the same as above, but for male patients Level 1^^ 
Female patient Level 1^^ 

History of osteoporosis or fracture 
No use of HRT, bisphosphonate, teriparatide, selective 
oestrogen receptor modulators or strontium 

Male patient Level 1!^^ 

History of osteoporosis or fracture 

No use of bisphosphonate or teriparatide 

Patient aged >65 years Level 1!!^^ 

History of osteoporosis 

Patient not receiving adequate levels of calcium and 
vitamin D 

Patient on high dose inhaled corticosteroid (>400 ^ig Level 1!^^ 

fluticasone daily or equivalent) for more than 1 year 
Bone mineral density not measured in the previous 
24 months 

Patient aged >65 years Level 1^'' 

Use of a falls-risk medicine (eg, long-acting hypnotic or 
anxiolytic, tricyclic antidepressant) 

Concurrent use of calcitriol with digoxin Level 111^® 

Calcium concentration not monitored in the previous 
3 months 

Use of calcitriol Level 1!!^^ 

Plasma calcium concentration not monitored in the 
previous 3 months 

History of diabetes Level II — monitoring. 

Microalbuminuria and plasma creatinine not monitored in Level I — ACE/ARB 
the previous 12 months use^° 
Patient not on ACE! or ARB 

NSAID use for >3 months Level 1!^^ 

Serum creatinine not monitored in the previous 12 months 

Use of lithium Level 1!!^^ 

Serum creatinine not monitored in previous 6 months 

History of BPH Level 1!!^^ 

Use of an anticholinergic agent 



Removed dose 



Removed dose® ^® 
Changed from history 
of fall^ 



Changed from history 
of fall^ 



Included all falls-risk 
medicines^"® 



Changed monitoring from 3 
to 12 months^® 



Continued 



Table 1 Continued 



Source of indicator 



Hospitalisation 
Number outcome 



Process of care (preceding hospitalisation) 



Level of evidence 



Previously 
published; not 
modified 



Previously published; 
modified for this study 



Newly 
developed 



42 



Urinary retention 



Diabetes indicators 
43 Hyperglycaemia/ 
hypoglycaemia 
Hypoglycaennia 



44 

I 

46 

47 
48 



Hyperglycaemia or 
hypoglycaemia 
Hyperglycaemia or 
hypoglycaemia 



Hypoglycaemia 
Cardiovascular disease 



1 . Use of two or more agents with anticholinergic activity OR 
use of a highly anticholinergic agent 



Use of an oral hypoglycaemic agent 

HbA1c level not monitored in the previous 6 months 

Use of a long-acting oral hypoglycaemic agent 

(glibenclamide or glimepiride) 

HbA1c level not monitored in the previous 6 months 

Use of insulin 

HbA1c level not monitored in the previous 6 months 
Use of insulin or oral hypoglycaemic medicines 
Use of medicines that may increase or decrease blood 
glucose concentration 

HbA1c level not monitored in the previous 6 months 

Use of glibenclamide or glimepiride 

Renal function not monitored in the previous year 

History of diabetes 

Not on lipid lowering drug 



Level II 

Level 1^^ 
Level 1^"^ 

Level 1^"^ 
Level 1^^ 

Level 1!^^ 
Level ir^ 



Combined to one indicator^ 



Added hypoglycaemia 

as outcome^ ^ 

Added HbA1c monitoring^ 



ARB, angiotensin receptor blocker; BPH, benign prostatic hyperplasia; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; Gl, gastrointestinal; HbA1c, glycated 
haemoglobin; HRT, hormone replacement therapy; IHD, ischemic heart disease; LABA, long-acting p agonist; MAOl, monoamine oxidase inhibitor; Ml, myocardial infarction; NSAID, 
non-steroidal anti-inflammatory drug; PPI, proton pump inhibitor; SSRI, selective serotonin reuptake inhibitor; TCA, transluminal coronary angioplasty; VTE, venous thromboembolism. 
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setting and 21 were newly developed. These were then 
sent to the expert clinical panel for full validation and 
were independently assessed by 78 expert clinicians 
(22.2% response rate). Of the respondents, 32% were 
medical physicians and 68% were pharmacists. 

The expert panel considered 29 of these (60.4%) to 
be sufficiently valid based on the a priori developed cri- 
teria (table 2). The majority of these (72.4%, n=21) 
were based on level I evidence. A total of 11 cardiovascu- 
lar indicators were developed, of which 5 (45.5%) were 
validated by the expert clinical panel; four of the five 
were based on level I evidence. Of the five mental 
health indicators developed, only one had level I evi- 
dence and none were validated by the expert panel. Six 
respiratory indicators were developed and five of these 
were validated, 80% of these were based on level I evi- 
dence. A total of seven gastrointestinal indicators, six of 
which were based on level I evidence were developed 
and five (71.4%) were validated. Of eight osteoporosis/ 
fracture indicators, half of which were based on level I 
evidence and five (62.5%) were validated. Only two of 
the five developed renal indicators were validated, with 
the level of evidence being level II or less for these. 
Finally, six diabetes indicators were developed, four of 
which were based on level I evidence and all were vali- 
dated by the clinical panel. 

DISCUSSION 

This study provides a set of face and content validated 
indicators of medication-related potentially preventable 
hospitalisations, specific for the Australian healthcare 
setting linking suboptimal processes of care and medi- 
cation use with subsequent hospitalisation. Of a poten- 
tial 48 developed indicators, 29 achieved consensus 
validation by the expert clinical panel and over 70% of 
these were based on level I evidence. An important 
feature of these developed indicators is that they are 
evidence based, systematically combining evidence- 
based recommendations from clinical guidelines with 
expert clinical opinion. In addition, these indicators 
focus on those chronic conditions which are included 
in Australia's National Health Priority Areas^^ or those 
that are associated with high disease burden in 
Australia.^^ 

For each of the six disease clusters for which indica- 
tors were developed, the proportion validated by our 
expert panel ranged from only 20% (1/5) for the 
mental health indicators to 100% for the diabetes indi- 
cators (6/6). Interestingly, the level of evidence available 
for the metal health indicators around medicine use 
and processes of care was minimal (four of the five indi- 
cators had only level III evidence), by comparison to the 
diabetes indicators where the majority of evidence was 
level I. 

The health conditions for which these indicators were 
developed significantly contribute to the burden of 
illness, social and financial costs in Australia, and 



prevention of hospitalisations associated with these con- 
ditions will provide significant gains in the health of 
Australia's population. Furthermore, given the high 
prevalence of medication-related hospitalisations in 
Australia, identification of areas where medication man- 
agement could be improved, particularly at the primary 
care level, may also lead to fewer hospitalisations. An 
estimated 90 000 hospital admissions annually are con- 
sidered to be potentially preventable mediation-related 
admissions in those aged 65 years and older. Based on 
the average cost of hospitalisation in Australia in 2010- 
2011 to be $A5400,^^ these unnecessary hospitalisations 
cost Australia's healthcare system $A480 million 
annually. 

Analysis of the developed indicators in a population- 
based sample is required to establish operational validity, 
and this will be the focus of the next phase of this 
research. With the advent of computerised administra- 
tive health databases, these indicators have been devel- 
oped with the potential to be analysed in such databases 
at the population level. Importantly, the characteristics 
of those patients' most vulnerable to mediation-related 
hospitalisations will also facilitate the identification of 
risk-factors associated with suboptimal medication man- 
agement. Implementation of these indicators within 
routine monitoring of the Australian healthcare system 
will serve to highlight those conditions where hospitalisa- 
tions could potentially be avoided through improved 
medication management, identif)^ areas of current prac- 
tice that may be suboptimal or evidence-practice gaps 
and facilitate the development of specific interventions 
to improve healthcare and subsequent patient 
outcomes. 

The standard RAND appropriateness method employs 
two rounds; in the first round experts rate indicators 
independently and in the second round, experts meet 
face to face to discuss the indicators and rate the indica- 
tors again, based on the face-to-face discussion. Our 
study used a modified RAND appropriateness method, 
with one round of independent expert panel review, 
and subsequent inclusion of indicators which met a 
priori defined criteria but no face-to-face meeting of 
experts. This is a potential limitation of our study, 
because the face-to-face meeting provides an opportun- 
ity to discuss indicators with low levels of agreement 
between experts, and can identify whether this is due to 
true clinical disagreement or simply an issue with the 
wording of the indicator. "^"^ It may be that for some indi- 
cators true consensus was not achieved if those who dis- 
agreed were strongly opposed to the indicator, and this 
type of issue may have been identified at a face-to-face 
meeting. Despite these limitations, the online survey 
technique used in our study eliminates any potential 
bias from dominant individuals who may be associated 
with face-to-face panel settings. "^"^ This allows for expert 
clinical panel members to express their opinions in an 
anonymous manner but also gives them time to consider 
each of the four elements of preventability together with 
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prior to admission) 
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History of bipolar disorder 
Use of lithium 
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Patient aged >65 years 
Use of 2 or more agents with 
anticholinergic activity OR use 
of an agent with high 
anticholinergic activity 
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Use of multiple psychotropic 
medications (eg, 
benzodiazepines, tricyclic 
antidepressants) 
Use of duloxetine, fentanyl, 
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Concurrent treatment with strong 
CYP1A2 inhibitors (eg, 
duloxetine) with fluvoxamine 

History of asthma or COPD 
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History of asthma 

Use of SABA more than 3 
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No use of inhaled 

corticosteroids 
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Asthma or COPD 



Influenza-related 
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Pneumococcal 
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sepsis 
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26 



27 



perforation or ulcer 
or gastritis 

Chronic 
constipation or 
impaction 

Chronic 
constipation or 
impaction 

Gl ulcer 

Gl ulcer 



Moderate-to-severe COPD with 
frequent exacerbation 
Use of long-acting p-agonist or 
anticholinergic 
No use of inhaled 
corticosteroids 
History of asthma or COPD 
No contraindication to influenza 
vaccine 

No influenza vaccination in the 
previous year 
Patient aged >65 years 
No contraindication to influenza 
vaccine 

No influenza vaccine in the 
previous year 
Patient aged >65 years 
No contraindication to 
pneumococcal vaccine 
No pneumococcal vaccine in 
the previous 6 years 



History of Gl ulcer or bleeding 
NSAID use for at least 1 month 
3) No use of gastroprotective 
agent (eg, PPI) 
Use of 2 or more agents with 
low-to-moderate anticholinergic 
activity; OR use of a highly 
anticholinergic agent 
Regular use of a strong opioid 
analgesic (fentanyl, oxycodone, 
morphine) 

No concurrent use of a laxative 
Patient with dyspepsia 
PPI not prescribed 
Patient with a positive test for 
Helicobacter pylori 
Not prescribed H pylori 
eradication therapy (PPI twice 
daily, clarithromycin 500 mg 
twice daily and amoxycillin 1 g 
twice daily for 7 days: OR PPI 



90 



90 



75 



100 



95 






Continued 



Table 2 Continued 



Would you expect most health professionals to* 



Hospitalisation 
Number outcome 



Process of care 
(preceding hospitalisation) 



Overall 
score 
Accepted (%) 



Recognise the 
problem in the 
process of 
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29 



Gl ulcer or bleed 



Oesophagitis, 
oesophageal 
ulceration or 
stricture 



Osteoporosis/fracture indicators 
30a Osteoporosis or 
fracture 




twice daily, clarithronnycin 
500 mg twice daily and 
nnetronidazole 400 mg twice 
daily for 7 days; PPI twice daily, 
amoxycillin 500 mg three times 
a day and metronidazole 
400 mg three times a day for 
1 4 days) 

Patient with osteoarthritis 
Dispensed long-term NSAIDs 
(including COX-2) therapy 
History of oesophageal 
disorders (active oesophagitis, 
oesophageal ulceration, stricture 
or achalasia) 
Use of alendronate 

Use of systemic corticosteroids 

for at least 3 months 

No osteoporosis prophylaxis 

(women: no use of HRT, 

bisphosphonate, teriparatide, 

selective oestrogen receptor 

modulators or strontium; men: 

no use of bisphosphonate or 

teriparatide) 

Female patient 

History of osteoporosis or 

fracture 

No use of HRT, 
bisphosphonate, teriparatide, 
selective oestrogen receptor 
modulators or strontium 
Male patient 

History of osteoporosis or 
fracture 

No use of bisphosphonate or 
teriparatide 

Patient aged >65 years 
History of osteoporosis 
Patient not receiving adequate 
levels of calcium and vitamin D 
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Hypercalcaemia 



Renal indicators 
38 Renal failure or 

^ nephropathy 



39 
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40 
41 



Renal failure 

Renal failure 

Urinary retention 
Urinary retention 



Diabetes indicators 
43 Hyperglycaemia/ 
hypoglycaemia 



1 . Patient on high dose inhaled 
corticosteroid (>400 |xg 
fluticasone daily or equivalent) 
for more than 1 year 
Bone nnineral density not 
nneasured in the previous 
24 months 

Patient aged >65 years 
Use of a falls-risk medicine (eg, 
long-acting hypnotic or 
anxiolytic, tricyclic 
antidepressant) 

Concurrent use of calcitriol with 
digoxin 

2. Calcium concentration not 
monitored in the previous 
3 months 

1 . Use of calcitriol 

2. Plasma calcium concentration 
not monitored in the previous 
3 months 

1 . History of diabetes 

2. Microalbuminuria and plasma 
creatinine not monitored in the 
previous 1 2 months 

3. Patient not on ACE! or ARB 

1 . NSAID use for >3 months 

2. Serum creatinine not monitored 
in the previous 12 months 

1. Use of lithium 

2. Serum creatinine not monitored 
in the previous 3 months 

1. History of BPH 

2. Use of an anticholinergic agent 
1 . Use of 2 or more agents with 

anticholinergic activity OR use 
of a highly anticholinergic agent 

1 . Use of an oral hypoglycaemic 
agent 

2. HbAlc level not monitored in 
the previous 6 months 
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41 




Continued 



Table 2 Continued 



Would you expect most health professionals to* 



Hospitalisation 
Number outcome 



Process of care 
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problem in the 
process of 
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associated with the 
process of care? (%) 
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the process of care to 
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process of care to 
reduce the likelihood 
of hospitalisation? (%) 



44 Hypoglycaennia 



46 



47 



48 



Hyperglycaemia or 
hypoglycaemia 

Hyperglycaennia or 
hypoglycaennia 



1. 



Hypoglycaennia 



Cardiovascular 
disease 



Use of a long-acting oral 
hypoglycaennic agent 
(glibenclannide or glinnepiride) 
HbA1c level not nnonitored in 
the previous 6 nnonths 
Use of insulin 

HbA1c level not nnonitored in 
the previous 6 nnonths 
Use of insulin or oral 
hypoglycaennic nnedicines 
Use of nnedicines that nnay 
increase or decrease blood 
glucose concentration 
HbA1c level not nnonitored in 
the previous 6 nnonths 
Use of glibenclannide or 
glinnepiride 

Renal function not nnonitored in 

the previous year 

History of diabetes 

Not on lipid lowering drug 



95 100 



91.5 100 



76.8 88 



90 



95 



75 



95 



90 



75 



95 



81 



69 



81.5 75 



81.8 88 



75 



88 



88 



88 



88 



63 



Nunnbers in bold represent those who achieved an average score of >70% agreennent by the expert panel. 
*Percentage of respondents who answered 'Agree' or 'Yes' on the three-point Likert scale. 

AF, atrial fibrillation; ARB, angiotensin receptor blocker; BPH, benign prostatic hyperplasia; CHF, congestive heart failure; COPD, chronic obstructive pulnnonary disease; Gl, gastrointestinal; 
HbA1c, glycated haennoglobin; IHD, ischennic heart disease; LABA, long-acting p agonist; MAOl, nnonoannine oxidase inhibitor; Ml, nnyocardial infarction; NSAID, non-steroidal anti-inflannnnatory 
drug; PPI, proton punnp inhibitor; SABA, short-acting p agonist; SSRI, selective serotonin reuptake inhibitor; TCA, translunninal coronary angioplasty; VTE, venous thronnboennbolisnn. 
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the supporting evidence base of each developed indica- 
tor. In addition, our method for developing the indica- 
tors systematically combined the available evidence base 
with the opinion of clinical experts to develop indicators 
that are both face and content valid/^ The modified 
RAND method used in our study has been used in indi- 
cator development studies previously/^ A recent 
Australian study used this method to validate 657 indica- 
tors of healthcare appropriateness/^ 

Our study achieved a 22% response rate, which is 
lower than other Australian studies involving medical 
practitioners, which typically achieve a response rate of 
around 30%/^"^^ While this may limit the generalisabil- 
ity of our findings, our results are strengthened by 
having medical specialists, general practitioners and 
pharmacists on the expert review panel/"^ In addition, 
78 expert clinicians reviewed the clinical indicators for 
our study; by comparison, previous studies which devel- 
oped clinical indicators for preventable medication- 
related hospitalisation used fewer than 20 expert 
reviewers. 

In conclusion, this study has developed a set of face 
and content validated indicators of medication-related 
potentially preventable hospitalisations specific for the 
Australian healthcare setting, linking medication use 
with suboptimal processes of care resulting in adverse 
outcomes of hospitalisations. As a measure of health 
system performance these indicators could identify areas 
of sub-optimal medication management, particularly at 
the primary care level, based on routinely collected 
health administrative health data but with the strong 
focus on patient outcomes and quality of care rather 
than processes or quantity. 
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